"Inplace" iterators over the coefficients, monomials, terms and exponent vectors of a multivariate polynomial #2196

joschmitt · 2025-10-24T13:42:28Z

Add an inplace option to the iterators of exponents, coefficients, terms and monomials of a multivariate polynomial.
This is still a draft. Is this the "interface" we want?
Regarding the new functions coeff!, term! and exponent_vector!: Should they be exported (I did it for now because monomial! is exported as well)? Do I need to implement an "abstract fall-back" in src/MPoly.jl?

Nemocas/Nemo.jl#2161 implements the (few) necessary functions for QQMPolyRingElem. Using those, I get the following timings in oscar-system/Oscar.jl#5483 :

# Previously
julia> @btime inv_c[n-k+1:n];
  3.628 s (28973402 allocations: 1.73 GiB)

# With inplace iterators
julia> @btime inv_c[n-k+1:n];
  1.356 s (2811 allocations: 279.24 KiB)

So that moves in the right direction.

joschmitt · 2025-10-24T13:47:58Z

src/generic/MPoly.jl

+   if !isassigned(m.coeffs, 1) || !is_one(m.coeffs[1])
+     m.coeffs[1] = one(base_ring(x))
+   end


This is a bit ugly, but it gets the monomials iterator to constant allocations. Otherwise it would allocate a new 1 all the time. Is this too much of a corner case?

Seems fine to me

codecov · 2025-10-29T14:16:18Z

Codecov Report

❌ Patch coverage is 90.90909% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.99%. Comparing base (995a497) to head (d62e1c5).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
src/MPoly.jl	55.55%	8 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2196      +/-   ##
==========================================
- Coverage   87.99%   87.99%   -0.01%     
==========================================
  Files         127      127              
  Lines       31769    31802      +33     
==========================================
+ Hits        27956    27983      +27     
- Misses       3813     3819       +6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joschmitt · 2025-10-29T16:04:18Z

src/generic/GenericTypes.jl

+# S may be the type of anything that can store an exponent vector, for example
+# Vector{Int}, ZZMatrix, ...
+struct MPolyExponentVectors{T <: AbstractAlgebra.RingElem, S}
   poly::T
+   inplace::Bool
+   temp::S # only used if inplace == true
+end


The exponent_vectors iterator can now in principle take anything as type for the exponents, as long as there is a exponent_vector!(::T, ...) method for that type. Now one can do exponent_vectors(Vector{ZZRingElem}, ::QQMPolyRingElem) and it would for example allow to add a variant exponent_vectors(ZZMatrix, ...) in Nemo like mentioned in oscar-system/Oscar.jl#5483 (comment) .

fingolfin · 2025-11-03T14:16:28Z

Thank you very much for working on this. It seems fine to me. But I'd like triage to have a look at it before merging.

src/MPoly.jl

lgoettgens · 2025-11-05T11:21:53Z

IMO this should also have the abstract fallbacks, see Nemocas/Nemo.jl#2161 (comment)

joschmitt · 2025-11-05T14:37:27Z

IMO this should also have the abstract fallbacks, see Nemocas/Nemo.jl#2161 (comment)

Also done.

lgoettgens

some more minor comments. I didn't really think about exponent vectors yet, mostly about the iterators that return some ring elem of some kind

lgoettgens · 2025-11-05T15:33:48Z

src/MPoly.jl

-function coefficients(a::MPolyRingElem{T}) where T <: RingElement
-   return Generic.MPolyCoeffs(a)
+function coefficients(a::MPolyRingElem{T}; inplace::Bool = false) where T <: RingElement
+   return Generic.MPolyCoeffs(a, inplace=inplace)


Suggested change

return Generic.MPolyCoeffs(a, inplace=inplace)

return Generic.MPolyCoeffs(a; inplace)

IMO a bit neater, but I don't really care

I never really got the hang of this syntax, so if you don't care, I'll prefer my version.

src/generic/GenericTypes.jl

lgoettgens

Looks great! Thanks for working on this!

joschmitt · 2025-11-10T16:14:31Z

src/MPoly.jl

+
+If `inplace` is `true`, the elements of the iterator may share their memory. This
+means that an element returned by the iterator may be overwritten 'in place' in
+the next iteration step. This may result in significantly fewer memory allocations.


I added these sentences to the doc strings. I'm happy for any suggestions. If I understand the suggestion in oscar-system/Oscar.jl#5530 (comment) correctly, we want to have something to add to all iterator doc strings that allow inplace. So maybe we should agree on a nice wording here and then we can copy and paste it everywhere :)

@fingolfin @mjrodgers

EDIT: I kept this intentionally vague ("may") because with this generic code it might be that the inplace version actually doesn't do anything different (because something is not implemented for a particular type).

I like this, if others agree I will add this same wording to the other methods in Oscar.

Maybe mention that it might to lead to unexpected behavior when calling collect on such an iterator.

I added a bit more, so this is ready for the next round of feedback :)

I feel like in an ideal world, we would put this paragraph somewhere "central" in the documentation and just link there from all iterators that support inplace = true. But I don't know where.

Looks good to me, thanks. (And I agree, one "central place" would be great, but for the time being this here is fine.)

joschmitt · 2025-11-14T09:05:41Z

I'll merge this beginning of next week, if there are no further comments.

Use `@inferred` correctly and fix return type

joschmitt mentioned this pull request Oct 24, 2025

Towards inplace iterators for exponents of multivariate polynomials etc. Nemocas/Nemo.jl#2161

Closed

joschmitt commented Oct 24, 2025

View reviewed changes

joschmitt force-pushed the js/inplace branch 2 times, most recently from 4f33ce7 to 7275f84 Compare October 29, 2025 13:40

joschmitt mentioned this pull request Oct 29, 2025

Don't use AbstractAlgebra internals oscar-system/Oscar.jl#5508

Merged

joschmitt force-pushed the js/inplace branch from 2bc128b to e577691 Compare October 29, 2025 15:20

joschmitt commented Oct 29, 2025

View reviewed changes

joschmitt marked this pull request as ready for review October 29, 2025 16:04

joschmitt closed this Oct 30, 2025

joschmitt reopened this Oct 30, 2025

fingolfin added the triage label Nov 3, 2025

joschmitt changed the title ~~Towards inplace iterators for exponents of multivariate polynomials etc.~~ "Inplace" iterators over the coefficients, monomials, terms and exponent vectors of a multivariate polynomial Nov 4, 2025

joschmitt added the release notes: use title For PRs: the title of this PR is suitable for direct use in the release notes label Nov 4, 2025

fingolfin reviewed Nov 5, 2025

View reviewed changes

src/MPoly.jl Outdated Show resolved Hide resolved

fingolfin reviewed Nov 5, 2025

View reviewed changes

src/MPoly.jl Outdated Show resolved Hide resolved

joschmitt force-pushed the js/inplace branch from 85224a4 to 18cde7d Compare November 5, 2025 14:35

lgoettgens reviewed Nov 5, 2025

View reviewed changes

lgoettgens approved these changes Nov 6, 2025

View reviewed changes

joschmitt added 8 commits November 10, 2025 17:10

Towards inplace iterators for exponents of multivariate polynomials etc.

ff7c58d

Add one-element constructors

c4ef7ca

Fix type signature

13bc030

Allow "arbitrary" types for exponent vectors

7ebd24f

Add a small test

9100d4f

Make it Nemo proof

8bd71ef

Only allocate when asked

c774ba0

Provide "abstract" fallbacks

49c5192

joschmitt added 5 commits November 10, 2025 17:10

Re-add the "default" constructor

3057fdc

Do it differently

4a0ae6d

Remove some constructors

7d73418

Increase the odds

ef3b593

Add some documentation

d6e24a0

joschmitt force-pushed the js/inplace branch from a597880 to d6e24a0 Compare November 10, 2025 16:10

joschmitt commented Nov 10, 2025

View reviewed changes

fingolfin removed the triage label Nov 12, 2025

Add more documentation

d62e1c5

lgoettgens merged commit 0a6e99f into Nemocas:master Nov 17, 2025
23 checks passed

joschmitt deleted the js/inplace branch November 17, 2025 09:40

joschmitt added a commit to joschmitt/AbstractAlgebra.jl that referenced this pull request Nov 20, 2025

Fixup Nemocas#2196

a5c3bba

Use `@inferred` correctly and fix return type

This was referenced Nov 20, 2025

Fixup #2196 #2221

Open

More inplace functions for the flint mpoly types Nemocas/Nemo.jl#2178

Open

joschmitt added a commit to joschmitt/AbstractAlgebra.jl that referenced this pull request Nov 24, 2025

Fixup Nemocas#2196

102ec7b

Use `@inferred` correctly and fix return type

	return Generic.MPolyCoeffs(a, inplace=inplace)
	return Generic.MPolyCoeffs(a; inplace)

"Inplace" iterators over the coefficients, monomials, terms and exponent vectors of a multivariate polynomial #2196

"Inplace" iterators over the coefficients, monomials, terms and exponent vectors of a multivariate polynomial #2196

Uh oh!

Conversation

joschmitt commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fingolfin commented Nov 3, 2025

Uh oh!

Uh oh!

Uh oh!

lgoettgens commented Nov 5, 2025

Uh oh!

joschmitt commented Nov 5, 2025

Uh oh!

lgoettgens left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lgoettgens left a comment

Choose a reason for hiding this comment

Uh oh!

joschmitt Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joschmitt commented Nov 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

joschmitt commented Oct 24, 2025 •

edited

Loading

codecov bot commented Oct 29, 2025 •

edited

Loading

joschmitt Nov 10, 2025 •

edited

Loading